Appl. No. 09/728,538 

Amdt. dated November 30, 2004 

Reply to the Office Action of June 30, 2004 

REMARKS/ARGUMENTS 

Reexamination and reconsideration of this application as amended is requested. 
By this amendment, Claims 1-3, 14-17, and 27, have been amended. After this 
amendment, Claims 1-36 remain pending in this application. 

Applicants would like to thank the Examiner for allowance of Claims 4-13, 18-26, 
and 28-36 

Claim Rejections - under 35 USC § 112, first paragraph, and 
under 35 USC § 112, second paragraph 

(1-3) The Examiner rejected Claims 1-3, 14-17, and 27 under 35 U.S.C. 1 12, first 
paragraph. The Examiner further rejected Claims 1-3, 14-17, and 27 under 35 U.S.C. 
1 12, second paragraph, because Claims 1 and 15, included indefinite language relating to 
"the location on the image of the form where the pen stroike was entered...". 

Applicants have amended Claims 1-3, 14-17, and 27 to more clearly and distinctly 
recite the present invention. Applicants believe that the previous amendment introduced a 
possible confusion which should be corrected in the presently amended claims. 

First of all, Applicants wish to explain an embodiment of the present invention to 
help clarify any confusion with the amended claims. According to an exemplary system, 
the system automatically identifies a form by detecting pen stroke data when a digitizing 
pen is used to enter data in data entry fields on a physical form placed on a support 
surface, such as on an electronic tablet or the like. The system pen stroke data includes 
content information and also includes location information that indicates the location on 
the physical form where the pen stroke data was entered. The system then selects one of 
a set of electronic images of forms that best matches the physical form used to enter the 
detected pen stroke data by: 

1) automatically determining the location of the data entry field boundary of each 



Page 21 of 30 



Appl. No. 09/728,538 

Amdt. dated November 30, 2004 

Reply to the Office Action of June 30, 2004 

data entry field in each of the electronic images of forms; and 

2) selecting the best match of the location information of the pen stroke data with 
the location of the data entry field boundary of each data entry field. 

More specifically, note that a data entry field boundary, in one example, can be 
defined by the system automatically calculating a box delimiting the boundary around a 
data entry field in each electronic image of a form. The location of this field bounding 
box for each data entry field can be compared with the location information of the pen 
stroke data to determine a best match. In one embodiment, the location information of 
the pen stroke data can also be defined by a box delimiting the boundary around the pen 
stroke data. In this case, the system can compare the location of the pen stroke data 
bounding box (such as using the geometry of the corner X, Y coordinates) with the 
location of the field bounding box for each data entry field in each electronic image of a 
form. 

This inventive implementation has the significant advantage that a user can 
conveniently scan an image of each form and provide this bit image to the system to 
create a collection of bit images of forms. A user uses a pen and tablet, for example, to 
enter pen stroke data on a physical form. The location information of the pen stroke data 
is used by the system to automatically identify which stored bit image of a form matches 
the physical form used by the user to enter the pen stroke data. This is done by the 
system automatically calculating data entry field locations of data entry fields found in 
the bit images of the electronic forms. The system compares and finds a best match of 
the location information of the pen stroke data with the location of a data entry field in 
one of the bit images of the stored electronic forms. In this way, the system 
automatically identifies the bit image of the stored electronic form that best matches the 
physical form used by the user to enter the pen stroke data. 

There is no need to specially create form templates with pre-entered location 
information for identifying the location of data entry fields in a physical form. The prior 
art, such as the cited U.S. patent application publication US2002/0146170 by Rom, 
suffers from such disadvantages by using pre-created templates with pre-entered 
(typically manually entered) location information for each data entry field in each form. 
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There is no need for a user to manually identify to the system which form is being 
used by the user. There is no need for forms to include special content information to 
identify the form to the system. Lastly, there is no need for a strict procedural sequence 
such as used by the prior art for first creating a collection of stored form templates (with 
pre-entered data entry field location information) in a system and then using this 
collection of pre-stored form templates for capturing pen stroke data on a physical form. 
That is, for example, the present inventive system can capture pen stroke data and 
associated location information while entering data on one or more physical forms. 
Then, the inventive system can receive and store (such as by a user using scanning 
equipment to electronically scan physical forms) a collection of scanned bit images of 
forms. The system then can identify the physical form(s) used for the entry of the pen 
stroke data by finding a best match to one of the stored bit images of forms. The system 
identifies the best match form based on best matching the location information of the pen 
stroke data with the system's automatically calculated location of each data entry field in 
the bit image of each stored form. This is a significant advantage of the presently 
claimed invention not available before. 

With respect to independent Claim 1, and dependent Claims 2-3 and 14, which 
depend from amended independent Claim 1 , the following clarifications apply. 

1) The pen stroke data includes location information that indicates the location on 
the form where the pen stroke data was entered. 

2) The form selecting means selects the best match of the location information of 
the pen stroke data with automatically determined location of a data entry field boundary 
of each of at least one data entry field in an electronic bit image of at least one form. 

Claim 2 adds the following new and novel feature. 

3) The form selection means compares the location of a pen stroke data boundary 
defined by the location information from the pen stroke data with the automatically 
determined location of a data entry field boundary of each of at least one data entry field 
in the electronic bit image of the at least one form. 
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Claim 3 adds the following new and novel feature. 

4) The different forms have the data entry fields located in unique different 
locations such that the system can be made more fault tolerant by placement of data entry 
fields on different forms in disparate locations. 

Claim 14 adds the following new and novel feature. 

5) The form selection means additionally uses the content information from the 
pen stroke data to identify the content entered into a form and that is unique to a 
particular form to select the electronic image of a form. 

Please also note that amended Claims 15-17 and 27 were amended consistent with 
the disucssion above with respect to amended Claims 1-3 and 14. These same remarks 
and arguments as discussed above apply to Claims 15-17 and 27 and will not repeated 
here. 



Support for the amended claim language, as discussed above, is found in the 
specification of the present patent application as originally filed. See for example 
paragraphs [0049], [0052], [0053] and [0061], and also see [0030] to [0033]. No new 
matter was added. The relevant cited text with added underline highlighting is provided 
below for quick reference. 



[0049] Once the collection of strokes has been closed, a box is defined 
which is the bounding box of all points in the collection of strokes. This 
bounding box may then be compared with boxes representing the fields of 
different forms to find the boxes that most closely match it. One 
embodiment of the form identification procedure finds the page for which 
the total distance between the corners of these bounding boxes and the 
field boxes on the form template is at a minimum. The following 
improvement defines a different method of form identification. Since the 
box surrounding the handwritten text may frequently be larger than the 
form field because tall characters or characters with descenders, such as g 
or q, will frequently stray outside the bounding box of the entry field. For 
this reason it would be good to have a box representing the text which 
would not extend beyond the edges of the field even if the user's writing 
occasionally strays outside of that region. 
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[0052] The next step is to identify fields on the forms. A form is a set of 
scanned images, each representing one page of the form. For each page of 
the form, find the complete set of the largest rectangular boxes that • 
contain no black pixels. This algorithm includes a threshold allowing it to 
ignore isolated islands of a few pixels that may be due to dirt on the 
scanner, stray marks on the form, etc. 

[0053] Once the form fields are identified, the next step is to match data to 
the form fields. This operation matches the bounding boxes of stroke 
collections (as described above) with bounding boxes of fields on the 
forms. For a given set of raw data (e.g. a file of ink stroke data), it is 
assumed that all matched fields will belong to a single form. 



[0061] The electronic capture of form data and its combination with an 
electronic form is preferably transparent to the user. If pen device 8 is an 
electronic pen that also writes in ink on the paper form, data can be 
captured when the user is filling out a paper form without requiring any 
additional effort from the user. In fact, the user might even be unaware 
that the tablet computer 2 is an electronic device. For example, as an 
untrained user fills out a paper form attached to its surface, digitizing 
device 5 captures pen stroke data without the user's knowledge. The 
sequence and location of the raw pen-stroke data is analyzed to determine 
which form was filled out and which field on each page was the intended 
field for the subsets of stroke data. The form/field identification method 
allows one to use the tablet computer 2 as if it were an ordinary clipboard. 
An individual can select one of several multi-page forms, attach it onto the 
clipboard without any special attention to its positioning, fill out fields in 
any order, skipping between pages at will, leaving fields blank, etc. and 
having no other interaction with the tablet computer 2. 



[0030] The process of identifying fields, and thereby identifying forms, 
begins by isolating blocks of writing and calculating a box around this 
writing for each of these blocks. Each block of writing is assigned to one 
of the fields in the total set of fields from all pages of all forms with the 
following requirements: first, no two overlapping blocks of writing can be 
assigned to the same field; second, all of the assigned fields must belong 
to the same form; and last, the linear distances between the corresponding 
corners of data bounding boxes and field bounding boxes should be 
minimized. 
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[0031] Processing overhead can be reduced as follows: if the text boxes 
and field bounding boxes perfectly match a form, then processing can 
terminate at that point and that form can be selected. 

[0032] The boundary box 1 1 is illustrated in FIG. 3. As can be seen, when 
the pen device 8 is used to enter data 7 via pen strokes on digitizing tablet 
5, the system determines the perimeter borders of the text data to create a 
boundary box 1 1 . Of course, the boundary box 1 1 is shown on this figure 
for illustrative purposes, but it is actually created in the host computer 
during post processing and does not appear on the form or the digitizing 
tablet . 

[0033] Matching the text boundary boxes to form fields would be 
performed as follows. In the preferred embodiment, the search for an 
optimal assignment (i.e. when the field is identified) is terminated when 
the sum of distances between fields and data blocks falls below a 
predefined threshold. Further, the search will be guided by the following 
principles which take advantage of field placement and assume that users 
will generally, but not necessarily always, fill out the form pages in the 
obvious order. The steps to accomplish this use the following general 
rules: 

The system which is the subject of U.S. patent application publication 
US2002/0146170 by Rom, on the other hand, utilizes form templates having field locating 
information and field type information that are stored in, and then accessed from, a form 
template database. See FIG. 5, with particular reference to 502, 506, 508, 510, and 512, 
and see paragraph [0025]. The data entry field location identifying information is defined 
and stored in the form template database to enable the identification of the location of a 
data entry field in an associated form. 



Please note that contrary to the characterization of Rom in the Office Action, on 
page 2, lines 8-11, with reference to paragraphs [0024] and [0025], Rom actually creates a 
bitmap image of data entered as part of a user entering data in a physical form. See Rom 
paragraph [0016], line 4. This data bitmap image is then processed and submitted to a 
forms identification process that utilizes a form template having defined entry field 
locations. 



The locations of data entry (as indicated in the bitmap image) are compared with 
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defined entry locations in stored electronic form templates. See Rom paragraph [0016], 
lines 6-13. For a more detailed description of the stored bitmap image of data in a form, 
see also paragraph [0018], lines 16-17, paragraph [0019], lines 10-15, and paragraph 
[0020], lines 1-5. This data bitmap image is what is being referred to in later paragraphs 
[0024] and [0025]. 

Therefore, it is understood from Rom that the data captured in a bitmap image is 
then processed and the location of the data in the bitmap image is compared to the defined 
locations for data fields in a stored form template. That is, Rom defines in advance the 
locations for data fields in a known form and enters this location information into a stored 
form template. Then, Rom scans in a bitmap image of data entered into a form and 
compares the location of the scanned data bitmap image with the defined locations for 
fields in a stored form template. See Rom paragraph [0025], lines 4-6, and lines 9-13. 
See also Rom FIG. 1 where the overall process clearly shows that the bitmap image 106 is 
captured from the table or scanner. Then, after image cleaning and processing, the form 
identification process 1 1 0 compares the location of user entered data in the processed 
bitmap image to defined location information for data entry fields of a known form where 
this defined location information is stored in a form template 1 12. That is, after input of a 
data image including entered data, the location of a datum located in the data image is 
compared with the location identifying information of the form field stored in the form 
template (i.e., identified by pre-specified and pre-stored form field boundaries stored in 
the form template database) to determine the existence of datum within the field 
boundaries of a form represented by a form template in a plurality of form templates in the 
database. Rom's form-template-driven system teaches a different system and form 
identifying process than the presently claimed system and method, as recited for amended 
independent Claims 1 and 1 5, and for all dependent Claims depending therefrom, 
respectively. 

As discussed above, the presently claimed system provides many advantages to 
users and operators of the system. There is no need to create and store form templates, 
and particularly there is no need to manually figure out and provide form field boundaries 
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specification stored in the database. This additional complexity is avoided by the 
presently claimed system, which provides significant added value and utility to users of 
the system. 

Therefore, in view of the amendments and remarks above, Applicants believe that 
since Rom does not teach, anticipate, or suggest, the presently claimed system as 
discussed above, and since Applicants' amendment places the claims in a more clear and 
distinct form in view of the discussion above which should overcome the rejections under 
35 USC 112, first paragraph, and 35 USC 1 12, second paragraph, Applicants believe that 
Claims 1-3, 14-17, and 27 are now in allowable form, and kindly request that the 
Examiner withdraw the rejections of Claims 1-3, 14-17, and 27, and further urge the 
Examiner to allow these claims. 

Allowable Subject Matter 

Applicants wish to acknowledge the Examiner has allowed Claims 4-13, 18-26, 
and 28-36. 

Conclusion 

The foregoing is submitted as full and complete response to the Official Action 
mailed June 30, 2004, and it is submitted that Claims 1-36 are in condition for allowance. 
Reconsideration of the rejections is requested. Allowance of Claims 1-36 is earnestly 
solicited. 

No amendment made was related to the statutory requirements of patentability 
unless expressly stated herein. No amendment made was for the purpose of narrowing 
the scope of any claim, unless Applicants have argued herein that such amendment was 
made to distinguish over a particular reference or combination of references. 
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Applicants acknowledge the continuing duty of candor and good faith to 
disclosure of information known to be material to the examination of this application. In 
accordance with 37 CFR §§ 1 .56, all such information is dutifully made of record. The 
foreseeable equivalents of any territory surrendered by amendment are limited to the 
territory taught by the information of record. No other territory afforded by the doctrine 
of equivalents is knowingly surrendered and everything else is unforeseeable at the time 
of this amendment by the Applicants and the attorneys. 

Applicants believe that no additional fee for claims amendment is currently due. 
However, the Commissioner is hereby authorized to charge any required fee for claims 
amendment to prevent this application from becoming abandoned, to Deposit Account 
50-1556. 

Additionally, a petition for a two month extension of time to file this Response 
has been attached to this Response. The Commissioner is hereby authorized to charge 
the extension fee for response of ($2 15.00), or if this fee amount is insufficient or 
incorrect, then the Commissioner is authorized to charge the appropriate fee amount to 
prevent this application from becoming abandoned, or credit any overpayment, to 
Deposit Account 50-1556. 

If the Examiner believes that there are any informalities that can be 
corrected by Examiner's amendment, or that in any way it would help expedite the 
prosecution of the patent application, a telephone call to the undersigned at (561) 
989-9811 is respectfully solicited. 



The Commissioner is hereby authorized to charge any fees that may be required 
or credit any overpayment to Deposit Account 50-1556. 
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In view of the preceding discussion, it is submitted that the claims are in condition 
for allowance. Reconsideration and re-examination is requested. 
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